Answering a call with client through a host

ABSTRACT

A client device may be used to initiate or answer calls using a host device connected to a network. For example, when a phone operating as a host device receives an incoming call from a network, the phone may communicate the presence of the incoming call to one or more client devices such as laptop, desktop, tablet, or wearable devices that are associated with the host device. When an answer call selection is made at a client, a call answered communication may be sent from the client device to the host device. The host device may then communicate with the network to connect the call to the host, and when the call is connected, the host may hand-off the call to the client that sent the call answered communication.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/475,390, entitled “ANSWERING A CALL WITH CLIENT THROUGH A HOST”,filed Sep. 2, 2014, which claims priority from and is a nonprovisionalapplication of U.S. Provisional Application No. 62/005,534, entitled“ANSWERING A CALL WITH CLIENT THROUGH A HOST” filed May 30, 2014, theentire contents of each are herein incorporated by reference for allpurposes.

The present application is also related to U.S. ProvisionalApplications:

62/005,550 entitled “ANSWER AND HOLD WITH CLIENT AND HOST” byRauenbuehler et al. (Ref. No. P23172USP1), filed May 30, 2014;

62/005,606 entitled “CLIENT APPLICATIONS COMMUNICATING VIA A USERTUNNEL” by Tung et al. (Ref. No. P23188USP1), filed May 30, 2014;

62/005,565 entitled “PROXIED PUSH” by Pollack et al. (Ref. No.P23053USP1), filed May 30, 2014;

62/005,336 entitled “SMS PROXYING” by Circosta et al. (Ref. No.P23192USP1), filed May 30, 2014;

62/005,505 entitled “MANAGING CONNECTIONS OF A USER DEVICE” by Schobelet al. (Ref. No. P23295USP1), filed May 30, 2014;

62/005,565 entitled “APPLICATION-LEVEL ACKNOWLEDGEMENTS” by Pollack etal. (Ref. No. P23189USP1), filed May 30, 2014;

62/005,586 entitled “MESSAGES WITH ATTENUATING RETRANSMIT IMPORTANCE” byPollack et al. (Ref. No. P23190USP1), filed May 30, 2014;

62/005,799 entitled “PROTOCOL SWITCHING IN INTER-DEVICE COMMUNICATION”by Prats et al. (Ref. No. P22319USP1), filed May 30, 2014; and

62/005,990 entitled “USER INTERFACE FOR PHONE CALL ROUTING AMONGDEVICES” by Coffman et al. (Ref. No. P23298USP1), filed May 30, 2014,which are hereby incorporated by reference for all purposes. The presentapplication is also related to U.S. Provisional Application 61/953,591entitled “DYNAMIC LINK ADAPTATION FOR IMPROVED LINK MARGIN,” by Liu etal., filed Mar. 14, 2014, which hereby incorporated by reference for allpurposes.

BACKGROUND

The present disclosure relates to relay of audiovisual (AV)communications between devices as part of a call.

With the continuing proliferation of different types of networkeddevices, a single user that in the past may have only had one computeror one phone may now have a desktop computer, a laptop computer, amobile phone, a tablet, networked wearable devices, networked homeappliances, and other networked devices. Managing communications in suchan environment is an increasingly complex task. Embodiments describedherein may include devices, systems, and methods for managing suchcommunications.

BRIEF SUMMARY

Embodiments can provide systems, apparatuses, and methods for answeringa call with a client through a host. Client devices and host devices areprovided. A client device may be used to initiate or answer calls usinga host device connected to a network. For example, when a phoneoperating as a host device receives an incoming call from a network, thephone may communicate the presence of the incoming call to one or moreclient devices such as laptop, desktop, tablet, or wearable devices thatare associated with the host device. When an answer call selection ismade at a client, a call answered communication may be sent from theclient device to the host device. The host device may then communicatewith the network to connect the call to the host, and when the call isconnected, the host may hand-off the call to the client that sent thecall answered communication.

Other embodiments are directed to systems, portable consumer devices,and computer readable media associated with methods described herein.

A better understanding of the nature and advantages of embodiments ofthe present invention may be gained with reference to the followingdetailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system composed of one or more clients,hosts (e.g., a companion device), and servers communicating messages andcalls among each other according to various embodiments.

FIG. 2 is a block diagram of system that may be used, at least in part,to facilitate relay for answering and placing a call according tovarious embodiments.

FIG. 3 is a diagram of a client device and an associated host deviceaccording to various embodiments.

FIG. 4 is a diagram of a plurality of client devices and an associatedhost device according to various embodiments.

FIG. 5 illustrates a signal flow for a remotely initiated call accordingto various embodiments.

FIG. 6 describes a method of relaying a call between a client device anda host device according to embodiments of the present invention.

FIG. 7 describes a method performed by a client device that acts as theend point of a communication being relayed by an associated host deviceaccording to embodiments of the present invention.

FIG. 8 describes a method where a call may be initiated by a clientdevice of a user to be made via a host device of the user according toembodiments of the present invention.

FIG. 9 describes a method for initiating a relayed call using a clientdevice according to embodiments of the present invention.

FIG. 10 describes a method for relaying, by a host, a call initiated bya client device according to embodiments of the present invention.

FIG. 11 shows a protocol stack 1100 for communicating data according toembodiments of the present invention.

DETAILED DESCRIPTION

Embodiments described herein relate to the use of a first device foraccessing a call or communication on a second device. In particular,embodiments relate to the use of a host device, such as a mobile phone,as a relay for accessing a call on a different client device, such as atablet, laptop, wearable device, or other networked device.

In one embodiment, a host device of a user can relay a phone call from asecond party to a client device of the user. For example, when a phoneoperating as a host device receives an incoming call from a network, thephone may communicate the presence of the incoming call to one or moreclient devices such as laptop, desktop, tablet, or wearable devices thatare associated with the host device (e.g., a user has paired the twodevices).

In another embodiment, a client device of a user can be used to initiala call to a second party via a host device of the user. For example,when an answer call selection is made at a client, a call answeredcommunication may be sent from the client device to the host device. Thehost device may then communicate with the network to connect the call tothe host. When the call is connected, the host may hand-off the call tothe client that sent the call answered communication.

For example, in certain embodiments, a cellular phone may have a set ofrelay settings selected. The phone may interact with an identity serviceto associate the phone with an identity or account, and to therebyidentify any number of client devices that are also associated with theidentity or account. When a call is received, the cellular phone mayautomatically notify, for example, an associated tablet and anassociated wearable device that share an identity with the phone. Whenthe answer call button is selected on the tablet, the cellular phone mayrelay the call to the tablet, and notify the wearable device that thecall has been answered. The cellular phone may continue to relay callinformation to and from the tablet as part of the call until the call istransferred back to the phone or disconnected. Thus, if a user is usinga tablet, laptop, or wearable device when a call comes in on the user'sphone, but the user does not know where their phone is, the user cananswer the call from another device, and either use the device for thecall, or use the device temporarily until the user finds the phone andtransfers the call to the phone. Any number of different client devicesmay be used as part of such a system, including networked tablets,watches, home appliances, and so on.

I. Introduction

Examples of systems, devices, and methods that may be used to implementcertain embodiments are described below, along with non-exhaustiveexamples of selected variations and alternative embodiments that may beimplemented.

A. System

FIG. 1 is a block diagram of a system 100 composed of one or moreclients, hosts (e.g., a companion device), and servers communicatingmessages and calls among each other according to various embodiments.FIG. 1 and other figures are merely illustrative of an embodiment orimplementation, or of aspects of an embodiment or implementationdisclosed herein, and should not limit the scope of any invention asrecited in the claims. One of ordinary skill in the art may recognizethrough this disclosure and the teachings presented herein othervariations, modifications, and/or alternatives to those embodiments orimplementations illustrated in the figures. FIG. 1 is one example of asystem which may use relay via a host or mobile device to answer a calland enable the call to be performed with a client or companion device asan endpoint of the call. The devices in system 100 can include hardwareand/or software elements.

In one embodiment, system 100 includes an identity managementinfrastructure 105 (i.e., one or more servers that implement an identitymanagement service, authorization service, and/or authenticationservice), content infrastructure 110 (i.e., one or more servers thatimplement a voice/video call service, a messaging service, and/or a pushnotification service), mobile device 115, companion device 120, userdevice 125, provider 130, provider 135, and communications network 140.As illustrated, identity management infrastructure 105, contentinfrastructure 110, mobile device 115, companion device 120, user device125, provider 130, and provider 135 are each capable of communicatingwith and through communications network 140 (representing the Internet,wide area networks (WANs), metropolitan area networks (MANs), local areanetworks (LANs), wireless area networks (WiLANs), radio access network(RANs), public switched telephone network (PTSN), etc., and/orcombinations of the same). As illustrated, mobile device 115 cancommunicate directly with companion device 120 without utilizingcommunications network 140.

Identity management infrastructure 105 may be implemented in variousembodiments using a single server computer system or may includemultiple server computer systems, web servers, application servers,networks, interconnects, and the like. In various aspects, identitymanagement infrastructure 105 provides management of individualentities, their authentication, authorization, and privileges within oracross systems, such as content infrastructure 110. Identity managementservices provided by identity management infrastructure 105 can includetechnologies and services such as Active Directory, identity providers,password managers, access control providers, single sign-on (SSO)services, OAuth, security token services, or the like.

In various embodiments, identity management infrastructure 105 maintainsinformation that authenticates the identity of a managed entity (such asa user, organization, and any associated devices, resources, services,applications, or the like). Identity management infrastructure 105 canverify that an entity is who/what it claims to be using a password,biometrics such as a fingerprint, a distinctive behavior such as agesture pattern on a touchscreen, challenge-response protocols, one-timepasswords (OTPs), 2-way authentications, and other techniques. Identitymanagement infrastructure 105 further can manage authorizationinformation that defines what operations an entity can perform in thecontext of a specific application, service, or resource. Someauthorizations may be based on a role, device type, application,application type, or the like associated with a managed entity. Usersare granted roles often related to a particular job or job function.Identity management infrastructure 105 can also manage descriptiveinformation about managed entities and how and by whom that informationcan be accessed and modified. As part of identity management, one ormore host devices may be identified and associated with one or moreclient devices, such that incoming calls to the host devices may berelayed to the client devices, and such that the client devices may beused to initiate relayed calls using the host devices.

In some embodiments, identity management infrastructure 105 createsdigital identities for managed entities encompassing, for example,entity identifying information (PII) and ancillary information. In oneaspect, a managed entity can have multiple digital identities and eachdigital identity can encompass multiple attributes. For example, a usercan have a user identifier (e.g., a phone number, e-mail, etc.) that islinked to multiple devices. In addition to creation, deletion,modification of digital identities, identity management infrastructure105 can manage ancillary entity data for use by services, such contentinfrastructure service 110.

In further embodiments, identity management infrastructure 105 can storecapabilities of each device associated with a user identifier. Examplesof device capabilities include whether a device includes a specific typeor version of hardware, whether a device includes a specific type orversion of software (e.g., operating systems or applications), whether adevice is capable of performing a specific function such as placing andreceiving phone calls or sending and receiving short message service(SMS)/multimedia message service (MMS) messages, whether a device iscapable of maintaining connections with other devices, or the like. Thelist of devices associated with a user can be sent to and stored at anyother device of that user, such as mobile device 115 and companiondevice 120 when associated with the same user identifier. Identitymanagement infrastructure 105 can determine and collect capabilities ofa device when it is registered and associated with the user identifier.Identity management infrastructure 105 can update the capabilities of adevice periodically, for example, when the device re-registers orcommunicates with one or more services managed by identity managementinfrastructure 105.

In various embodiments, identity management infrastructure 105 canreceive a single user identifier, which is used to determine deviceidentifiers for devices associated with the user identifier. The deviceidentifier for each device can be unique relative to other devices of auser. During entity registration, in order to access services orresources managed by identity management infrastructure 105, one or moreuser or other identifiers and a unique entity or device identifier (UID)may be combined to generate an entity or device token.

In various embodiments, the token is encrypted by applying a hashingalgorithm (e.g., SHA-0, SHA-1, SHA-2, MD5, Whirlpool, or other hashingalgorithms). The token generated and encrypted for an entity can remainconstant in various embodiments. Once a token has been generated andencrypted by identity management infrastructure 105, the token can besent back to the entity. The entity in some aspects can then distributethe token to services or resources managed by identity managementinfrastructure 105 or other third party services for a variety ofpurposes relating to authentication, authorization, accounting, or thelike of the entity at those managed services or resources or the trusteddelivery of content to the entity by the third parties.

Content infrastructure 110 may be protected by and/or accessible toentities managed by identity management infrastructure 105. Contentinfrastructure 110 may be implemented in various embodiments using asingle server computer system or may include multiple server computersystems, web servers, application servers, networks, interconnects, andthe like.

Content infrastructure 110 can provide content to mobile device 115,companion device 120, and user device 125 as well as to other devicesand entities. Examples of content include a text message, a multimediamessage, an impending calendar event, an audio/video call (e.g., usingVOIP), or a notification of new data on a remote server. In oneembodiment, the content can originate from one or more sources managedby identity management infrastructure 105 or provided directly bycontent infrastructure 110. In other embodiments, the content canoriginate from other sources. For example, content may originate fromany one of mobile device 115, companion device 120, user device 125, andproviders 130 and 135.

In another example, content may be received from other sources such asthe Internet, cellular networks, public switched telephone networks, andthe like. Content infrastructure 110 can then route the content tomobile device 115, companion device 120, user device 125, and providers130 and 135. In one embodiment, content infrastructure 110 may routethrough the infrastructure an SMS message received from or destined to acellular network. In another embodiment, content infrastructure 110 mayroute through the infrastructure a voice call received from or destinedto a public switched telephone network.

In some embodiments, the content sent to mobile device 115 can beforwarded to companion device 120 for delivery to mobile device 115.Companion device 120 can also act and send signals on behalf of mobiledevice 115. In these embodiments, companion device 120 acts as a main orintermediary device and mobile device 115 acts as a proxied device.Content infrastructure 110 can coordinate how and whether companiondevice 120 can act and send signals on behalf of mobile device 115.

In some embodiments, content infrastructure 110 can send content to morethan one device, when appropriate. A user may be associated with bothmobile device 115 and companion device 120. Content infrastructure 110may route the content to both mobile device 115 and companion device120, such as to have a VOIP phone call ring on both devices or to have amessage appear in the inbox of the same application installed on bothdevices. In other embodiments, content is sent to only one device, e.g.,to companion device 120, which may forward a call to mobile device 115.When a call is being forwarded to a device, a phone number can identifywhich device is to receive the phone/video call, and that device canrelay a call as appropriate.

In one aspect, content can include of one or more pieces of data, suchas a device identifier (or token) as discussed above and a payload. Adevice token can be provided in content originating from a provider(e.g., provider 130 and/or 135), a device of a same user (e.g., fromeither mobile device 115 or companion device 120), or a device ofanother user (e.g., user device 125), together with any payload theprovider seeks to have delivered using content infrastructure 110. Thedevice token can contain information that enables content infrastructure110 to locate a device on which a particular service or clientapplication is installed and that is registered to receive the content.The payload may include new information received at a server applicationor a reference to where the information is to be found. The payload mayfurther include a property list that specifies how the user is to bealerted about this new information by the particular service or clientapplication.

An alert can come in a variety of forms. In one example, content can bedisplayed to a user as an alert message or other visual representation,such as a badge associated with an application icon. Availability of thecontent further can be announced by playing a sound when an alert orbadge is shown. When a user is notified that an application or servicehas a message, event, or other content data for them, they can launchthe application or service and see the details by either viewing thecontent, viewing information contained in a push notification, havingthe client application retrieve the referenced information, or the like.The user can also choose to ignore the notification, in which case theapplication is not activated.

As alluded to above, content infrastructure 110 can include pushnotification services that in addition to or in the alternative ofrouting content implement mechanisms to give client applications of pushproviders that are on user devices the ability to let users know thatnew content is available at one or more server applications, is on thedevice, or is incoming. A push provider (or simply provider) as usedherein can refer to an entity having information to be forward and/ordelivered using a push notification infrastructure. Generally, softwaredevelopers (acting as providers) originate notifications in their serversoftware when new data is available for users. A provider connects itsserver software with content infrastructure 110 through a persistent andsecure channel. Identity management infrastructure 105 can ensure thatthe provider is authenticated (e.g., that the provider is who theprovider alleges to be) and authorized to connect and utilizes contentinfrastructure 110 in a trusted manner.

While monitoring for incoming data intended for its client applications,when new data for an application arrives, the provider prepares andsends in one aspect a notification through its channel connection tocontent infrastructure 110, which pushes the notification to a pushconsumer or destination target device. Identity managementinfrastructure 105 can also ensure that the consumer or destinationtarget device is authenticated and authorized to connect to and utilizesservices of content infrastructure 110 in a trusted manner. A pushconsumer (or simply consumer or destination) can refer to an entitydesignated to receive information forwarded and/or delivered usingcontent infrastructure 110. Although the above describes a provider asthe originator of content or a notification of available content for thesake of simplicity, a provider in one instance may in turn become aconsumer in another, and vice versa. Additionally, mobile device 115 maybe a provider of content to companion device 120, and vice versa as wellhas provider 130 providing content to provider 135, and vice versa.

In one example of operation of content infrastructure 110, one or moreserver computers provide, provision, manage, and otherwise operate thepush notification service for propagating information between provider130, provider 135, mobile device 115, companion device 120, and userdevice 125. Each may establish at least one persistent connection (e.g.,an accredited and encrypted Internet protocol (IP) connection) withcontent infrastructure 110 to originate and/or receive content over thispersistent connection. As noted above, each and their connections can beauthenticated and authorized by identity management infrastructure 105.

If a notification delivered by content infrastructure 110 for anapplication associated with a user's device arrives when the applicationis not running, the user's device may alert the user that theapplication has data waiting for it as discussed above. Contentinfrastructure 110 may also provide a default quality-of-servicecomponent that provides store-and-forward capabilities. If contentinfrastructure 110 attempts to deliver a notification but a targetdevice is offline, the notification can be stored for a limited periodof time, and delivered to the device when it becomes available. In someembodiments, all recent notification for a particular application isstored. In some embodiments, only one recent notification for aparticular application is stored. For example, if multiple notificationsare sent while the device is offline, each new notification causes theprior notification to be discarded. This behavior of keeping only thenewest notification is referred to as coalescing notifications. In otherembodiments, if the device remains offline for a long time, anynotifications that were being stored for it may be discarded.

Provider 130 and provider 135 may be implemented in various embodimentsusing a single server computer system or may include multiple servercomputer systems, web servers, application servers, networks,interconnects, and the like. In various aspects, provider 130 andprovider 135 provide client applications that run on mobile device 115,companion device 120, and user device 125 and server applications thatprovide one or more services to which the client applications canconnect. Provider 130 and provider 135 may seek to notify the clientapplications accessible to one or more of mobile device 115, companiondevice 120, and user device 125 that information is available to theirrespective users.

In one aspect, a push provider is a software developer, company, ororganization that maintains server software configured to interact withone or more client applications on one or more of mobile device 115,companion device 120, and user device 125. Provider 130 and provider 135each connect with content infrastructure 110 through a persistent andsecure channel while monitoring incoming data intended for their clientapplications. In one embodiment, provider 130 and provider 135 connectover a binary interface that provides a high-speed, high-capacityinterface, e.g., using a streaming TCP socket design in conjunction withbinary content. The binary interface may be synchronous or asynchronous.For each interface, TLS (or SSL) may be used to establish a securedcommunications channel.

Mobile device 115, companion device 120, and user device 125 may be eachembodiment as a single device, a single computer system, multipledevices, or multiple computer systems. In various aspects, mobile device115, companion device 120, and user device 125 although labeleddifferently for convenience can each be embodied as a mobile device, awearable device, or other mobile device (e.g., a laptop, palmtop, mobilephone, smart phone, multimedia phone, portable media player, GPS unit,mobile gaming systems, etc.). In addition to or in the alternative,companion device 120 and user device 125 can be embodied as describedabove as well as being embodied as personal computer systems,mainframes, server computer systems, cloud services, or the like. Mobiledevice 115, companion device 120, and user device 125 may include avariety of technologies that provide a communications connection. Someexamples of connection technologies include wired connections (e.g.,Ethernet, fiber, digital subscriber line (DSL), etc.) and wirelessconnections (e.g., WiFi, Bluetooth, WiMax, 3G, 4G, LTE, etc.).

In one aspect, mobile device 115, companion device 120, and user device125 host one or more of a variety of client applications thatcommunicate with one or more server applications provided by one or moreproviders (e.g., providers 130 and 135). These client applications mayinclude applications specific to the intended function of a device (suchas telephony applications or GPS applications) as well as e-mailclients, update/upgrade clients, news clients, web/blog clients, podcastclients, social networking clients, or other types of clientapplications where notification messages may be sent. These clientapplications may represent to a user one or more notification messagesreceived using content infrastructure 110. Notifications can berepresented to users in one or more manners defined by an operatingsystem of the device, a graphical user interface toolkit, and/or theapplications themselves. Some examples of representations ofnotifications include a new e-mail indicator, a new news item indicator,a new podcast indicator, a change of on-line status of a socialnetworking friend, and the like. In various embodiments, another serviceoperating on a device can handle notifications for client applications.

As discussed above, mobile device 115, companion device 120, and userdevice 125 may receive an identifier (or device token) when a clientapplication initially connects with content infrastructure 110 in orderto receive push notifications. Providers 130 and 135 can use the token,or include the token, with any content or notification message so thatit can be appropriately forwarded back to the device using contentinfrastructure 110. In various embodiments, to ensure trust, a providercommunicates the token every time it connects with contentinfrastructure 110. Content infrastructure 110 can decrypt the devicetoken and validate using identity management infrastructure 105 that thetoken was generated for the destination device. To validate in oneembodiment, content infrastructure 110 ensures that the deviceidentifier contained in the token matches the device identifier in adevice certificate used when the device registered with identitymanagement infrastructure 105.

Referring to an operation of system 100 illustrated in FIG. 1, in oneembodiment, the operation can be to forward or otherwise communicate anotification message from provider 130 to companion device 120 asillustrated by path 145. In various embodiments, provider 130 sends anauthentication Secure Sockets Layer (SSL) certificate upon an initialconnection with content infrastructure 110. Identity managementinfrastructure 105 can authenticate and authorize provider 130 as aregistered and authorized sender of push notifications. This SSLcertificate can also be configured with additional user-defined data.Identity management infrastructure 105 can utilizes the additionaluser-defined data to identify provider 130 in a trusted fashion. Othersecure communications protocols (e.g., cryptographic protocols such asTransport Layer Security (TLS), etc.) can be used in other embodiments.

In some embodiments, where provider 130 is associated with a particularapplication (e.g., Email, Facebook, or Twitter) and includes additionalidentifying (e.g., user-defined) data within the SSL certificate,identity management infrastructure 105 can not only authenticateprovider 130, but also automatically provision push service for provider130 and the application utilizing content infrastructure 110. In otherwords, identity management infrastructure 105 can automatically extractany additional identifying data from the authentication certificate andhave content infrastructure 110 attach the additional identifying data(or a portion of the data) to content (e.g., push-notificationmessages). In some embodiments, the additional identifying data mayidentify a topic or feed associated with provider 130 (or an applicationof provider 130) to which a user might subscribe via contentinfrastructure 110. Thus, the additional information in theauthentication certificate can be leveraged to direct content to devicesthat have subscribed to the topic/feed or requested informationregarding the topic/feed. In this way, push service is automaticallyprovisioned for provider 130.

Once provider 130 is trusted, content infrastructure 110 receives thenotification message from provider 130. As discussed above, thenotification message may include a device token. Having received thenotification message from provider 130, content infrastructure 110determines the destination for the notification message. In variousembodiments, the destination is determined based on the device tokenthat is sent along with notification message. In some embodiments, it isnot necessary to send destination information as part of a token. Bydetermining or extracting the destination from the device token orotherwise obtaining destination information for the content, contentinfrastructure 110 can then determine whether the destination is“online” or otherwise accessible.

If the destination is online, in one embodiment, content infrastructure110 may then route the notification message to the destination companiondevice 120 illustrated by path 150, for example, via a persistentconnection maintained by companion device 120 with contentinfrastructure 110. If the destination is “offline” or otherwiseinaccessible to content infrastructure 110, the content may be storedand delivery retried at a later time. Content infrastructure 110 can inaddition to or alternatively route the notification message to mobiledevice 115 illustrated by path 155, for example, via a persistentconnection maintained by companion device 120 with contentinfrastructure 110. Content infrastructure 110 thus can route content toa single device, multiple devices at the same time, or to one device fordelivery to another device.

B. Content Infrastructure

FIG. 2 is a block diagram of system 200 that may be used, at least inpart, to facilitate relay for answering and placing a call according tovarious embodiments. In particular, IDS services as described herein,including IDS 205, may be used to facilitate discovery and communicationbetween a host device used as a relay and a client device used as a callendpoint. As recited herein mobile device 115 of FIG. 1 may beconsidered a host device, and companion device 120 may be considered aclient device for the purposes of a relay system. System 200 may be usedin a relay system and may be implemented in various embodiments using asingle server computer system or may include multiple server computersystems, web servers, application servers, networks, interconnects, andthe like. System 200 can be embodied as content infrastructure of FIG. 1in various embodiments.

In particular, FIG. 2 illustrates various examples of forwarding content(e.g., notification messages and phone/video calls) between devices,e.g., between providers and mobile devices, or between a sending deviceof one user and receiving devices of another user). In these examples,system 200 is shown with identity services (IDS) 205 having interface210 and identity management server (IMS) 215 and push notificationservices (PNS) 220 having provider interface 225, gateway 230 havingpresence information 235, device interface 240 having connectioninformation 245, and user device 250. Each service may be implementedusing hardware and/or software elements.

In one aspect, IDS 205 may be embodied as or form part of identitymanagement infrastructure 105. IDS 205 may be implemented in variousembodiments using a single server computer system or may includemultiple server computer systems, web servers, application servers,networks, interconnects, and the like. Interface 210 can enable anentity (e.g., mobile device 115 or provider 130) to connect (e.g., via anetwork) in order to take advantage of service provided by IDS 205.Interface 210 may incorporate load balancing and other connectionmanagement techniques allowing entities to communicate with Identitymanagement server 215.

In one embodiment, an entity sends information such as an authenticationcertificate that is received via interface 210 upon an initialconnection to IDS 205 or to a service, resource, or application managedby IDS 205 (e.g., PNS 220). Identity management server 215 canauthenticate and authorize a device, user, or organization sending theinformation as a registered and authorized entity. One or more types ofservices can be authorized or provisioned for the device, user, ororganization (e.g., call services, instant messaging services, chatservices, notification services, etc.). To support a security model forPNS 220, entities and their devices may be required to possess certaincertificates, certificate authority (CA) certificates, or tokens.

In one embodiment, each provider of content uses a unique providercertificate and private cryptographic key for validating theirconnection with PNS 220. This certificate can be provisioned by identitymanagement server 215 and identify the provider and/or a particulartopic published by the provider. In general, the topic is a bundle ID ofa client application. The provider may optionally wish to validate theservice, to which the provider is connected, using a public servercertificate provided by PNS 220. In various aspects, the provider usesthe public server certificate passed to it by identity management server215 when registering to authenticate the service to which the providerhas connected.

Identity management server 215 may also issue to each device, whichdesires to receive content, a unique private key and certificate thatthe device uses to authenticate itself to identity management server 215and establish a connection to PNS 220. A device usually obtains a devicecertificate and key from identity management server 215 during deviceactivation and stores them in a keychain. The device also holds itsparticular device token, which it receives during the service connectionprocess. Each client application that utilizes PNS 220 is responsiblefor delivering this token to its content provider.

Identity management server 215 may store any necessary certificates, CAcertificates, and cryptographic keys (private and public) for validatingconnections and the identities of providers and devices.

In this example, once the entity is trusted, system 200 allows theentity to utilize push notification services provided by PNS 220. PNS220 may be implemented in various embodiments using a single servercomputer system or may include multiple server computer systems, webservers, application servers, networks, interconnects, and the like. Theentity may be a provider or other notification provider desiring toconnect with PNS 220 (e.g., via a network). As alluded to above, in oneembodiment, provider interface 225 provides a high-speed, high-capacityinterface allowing push notification providers to communicate with PNS220. Provider interface 225 may incorporate load balancing and otherconnection management techniques allowing entities to communicate withPNS 220. Although provider interface 225 is shown as being linked togateway 230, provider interface 225 may be incorporated into gateway 230or device interface 240. As discussed above, a user device can be aprovider of content in various embodiments as well as be a destinationof content routed using PNS 220.

Gateway 230 may be implemented in various embodiments using a singleserver computer system or may include multiple server computer systems,web servers, application servers, networks, interconnects, and the like.Gateway 230 can determine the destination of content (e.g., pushmessages or call messages) received via provider interface 225 or deviceinterface 240. In various embodiments, gateway 230 can determine adestination based on presence information 235. In one aspect, presenceinformation 235 is maintained using a device's push token. Accordingly,when a push notification is received at gateway 230 directed to aparticular push token, gateway 230 can perform a lookup to determinewhether there is a TCP socket descriptor associated with that pushtoken. The socket descriptor can provide the TCP socket information andother networking information needed to transmit the push notification.In various aspects, presence information 235 includes mappings betweenauthenticated entities and their connections to PNS 220. Theseconnections can be utilized by PNS 220 for delivering content,notifications, and the like or otherwise communicating with an entity.Each mapping may be indicative of at least one entity and at least oneconnection mechanism to that entity, such as a network socket connectionor other connection identifier. For example, a mapping may identify adestination device by its device token or a provider by its provideridentifier. Additional information may be included in each mapping inorder to facilitate communication with the entity's device.

In some embodiments, in order to scale handling of connections from anincreasing number of users, devices, and providers utilizing services ofPNS 220, device connections in presence information 235 (or the devicesthemselves) may be managed according to at least one grouping or logicalpartition called a zone. Functions performed by gateway 230 may bepartitioned out to multiple servers that are assigned dynamically tohandle these groupings or zones. For example, one or more servers mightmanage, for a period of time, delivery to destinations assigned to onezone and then be switched, or reconfigured, to manage the delivery ofnotifications to destinations assigned to a different zone at a latertime. Each of these servers may also include routing information that isused to route content to other servers associated with a particular zoneof the destination of the content. Thus, when content is received at oneserver, another server designed to handle a predetermined zone isdetermined and the content can be forwarded to the appropriate server.In one aspect, functions performed by gateway 230 may be partitioned outto multiple servers to handle corresponding device connections (e.g.,device interface 240).

In various embodiments, gateway 230 is linked to device interface 240.Device interface 240 provides an interface to communicate with userdevice 250. Device interface 240 may incorporate load balancing andother connection management techniques allowing devices to communicatewith PNS 220. Although device interface 240 is shown as being linked togateway 230, device interface 240 may be incorporated into gateway 230or provider interface 225.

Device interface 240 in these examples allows presence information 235to be generated when device interface 240 is connected to user device250. User device 250 can assert its presence to PNS 220 uponestablishing a persistent connection. Device interface 240 thengenerates a device/connection mapping in connection information 245.Device interface 240 can back-propagate connection information 245 togateway 230 allowing gateway 230 to generate a device/connection mappingin presence information 235. In one aspect, presence information 235includes a device/courier mapping or link allowing gateway 230 todetermine an appropriate courier that acts as device interface 240connected to user device 250. The courier utilizes connectioninformation 245 (including any device/connection mappings or links)allowing the courier to determine connection information specific touser device 250 that can be used to deliver content to user device 250.In another aspect, presence information 235 and connection information245 may be substantially identical in that they include correspondencesbetween a given device and its connection with PNS 220.

In various embodiments, a device wishing to receive content via PNS 220sends authentication information either upon an initial connection withdevice interface 240 or directly to IDS 205. Identity management server215 can receive the authentication information either directly orindirectly and then authenticate and authorize the device or itsassociated user or organization as a registered and authorized entity.Once the device is trusted, PNS 220 is informed and PNS 220 thereaftermanages any connections made between the device and PNS 220 (such aswith device interface 240 in connection information 245). Deviceinformation available at device interface 240 in connection information245 can be periodically back-propagated to gateway 230 to generate orupdate presence information 235.

When the device initially connects with PNS 220, PNS 220 provisions thedevice. In various embodiments, a zone is provisioned for the device asalluded to above. Despite a particular zone assignment for each device,devices may lose their connection with device interface 240 for variousreasons. For example, a connection might be lost due to loss of cellularsignal, or wi-fi signal, loss of power, or because a mobile device haschanged geographic locations, etc. In other aspects, a connection may beintermitted as opposed to being persistent in order to conserve power orachieve other efficiency metrics.

When user device 250 attempts to reconnect to PNS 220, user device 250can connect with any courier acting as device interface 240. Inembodiments where device connections are assigned to at least onegrouping or zone, device interface 240 may provision a connection withone or more servers of gateway 230 that are assigned to handle the zoneof a connecting device. For example, if device interface 240 isconnected to user device 250 that is assigned to zone 1, then deviceinterface 240 can provision a connection with one or more serversresponsible for managing zone 1. Device interface 240 may thenback-propagate device information for user device 250 to the one or moreservers responsible for managing zone 1. In similar fashion, deviceinterface 240 may make connections with servers of different zones toback-propagate specific device information for devices associated withthose respective zones ensuring that no matter where or how user device250 connects to PNS 220, presence information 235 is up to date andavailable to determining how to route the content. In some embodiments,device interface 240 can be specific to a wireless carrier or internetservice provider (ISP) allowing PNS 220 to support the protocols orphysical connections specific to multiple third party entities.

According to one example, when gateway 230 receives content fromprovider interface 225, gateway 230 forwards the content received fromprovider interface 225 to device interface 240 based on its mappings inpresence information 235. Device interface 240 can deliver the contentreceived from gateway 230 to user device 250 for which information abouta persistent connection is maintained in connection information 245.

Upon receiving content from gateway 230, device interface 240 canperform a lookup or otherwise consult its device connections inconnection information 245 and send the content received from gateway230 to the appropriate device, for example, over the persistentconnection associated with user device 250. In one aspect, deviceinterface 240 inspects the device token associated with the content tobe delivered and determines whether a match is found between the devicetoken and the connections that device interface 240 manages inconnection information 245. Device interface 240 can deliver the contentusing the connection established by the device having the given devicetoken.

In one example of operation, user device 250 subscribes to a particularapplication managed by a provider and desires to receive notificationmessages for that application via PNS 220. Thus, user device 250 callsthe provider either directly via a communications network or utilizingPNS 220 and transmits its device token to the provider. The device tokenor its transmission may include not only a device's identificationinformation but may include an encrypted combination of a device's UIDand its zone identifier allowing PNS 220 to provision connectioninformation for the device according to the appropriate resourcesallocated to the zone.

When the provider sends a notification message to the particularapplication on user device 250, the provider connects to PNS 220 usingprovider interface 225 and sends the message to gateway 230. Even ifuser device 250 is associated with a particular zone, the provider doesnot need to connect to any particular gateway of PNS 220 to successfullypush a notification message to user device 250. For example, if gateway230 receives content from provider interface 225 and the content has adevice token, gateway 230 will look at the token and either route themessage to an appropriate server of PNS 220 (which may route the messageto device interface 240 or another courier of PNS 230) or route themessage directly to device interface 240.

If gateway 230 is the designated gateway, gateway 230 sends/forwards themessage to device interface 240 based on its device/courier mapping inpresence information 235 in some embodiments. Device interface 240 isthen able to lookup its connections in connection information 245 andsend the message to the device over the persistent connectionestablished by the device with device interface 240. In summary, incases where PNS 220 receives a message having a particular destination,a gateway of PNS 220 forwards that message directly to an appropriatecourier of PNS 220 using a device/courier mapping that was establishedwhen a device connects to PNS 220. In further embodiments, gateway 230can send/forward the message directly to user device 250 based on itsdevice/connection mapping in presence information 235. Gateway 230 cangenerated this mapping information from various sources to each of whicha device has established a connection.

II. Host and Client Device

FIGS. 3 and 4 illustrate client devices and host devices that may beused in conjunction with or as part of systems 100 and 200 of FIGS. 1and 2 to implement relay for answering a call. While two embodiments areillustrated by these figures, various implementations may include anynumber of client devices, host devices, and different networks which maybe supported by the devices in a particular setup.

It should be apparent that the systems shown in FIGS. 3 and 4 illustrateone embodiment, and that other similar embodiments can have more orfewer components than what is shown. Other embodiments may also includedifferent configurations of the illustrated components, while remainingwithin the scope of the innovations herein. The various components ofhost device 450 in particular as shown in FIG. 4 can be implemented inhardware, software, or a combination of both hardware and software,including one or more signal processing and/or application specificintegrated circuits.

FIG. 3 illustrates an embodiment with client device 300, host device350, and network 390. Client device 300 may be any device accessible bya wireless interface, including a desktop computer, a laptop computer, asmartphone, a tablet computer, a wearable device (e.g. a network enabledwatch, earpiece, or necklace) a networked appliance (e.g. a networkenabled refrigerator or clothes washer) media player, personal digitalassistant, key fob, access card, multi-function device, game system, orany other such client device. Client device 300 comprises memory 304,processor 302, input device 306, display 308, and wireless interface312. Embodiments of a client device capable of functioning as anendpoint of a call may additionally include speaker and microphonedevices as part of an audio unit. Host device 350 includes processor352, memory 354, input device 356, display 358, audio device 370,wireless interface 362, wireless interface 364, and baseband device 359.

Host device 350 may be any communication device capable of relaying anAV signal from a network 390 to a client device 300. Certain embodimentsof host devices may have multiple communications interfaces, such thatas part of the relay performed by a host device, the host devicecommunicates with a network using one communication interface based on afirst communication protocol, and a the host device communicates with aclient device using a second communication interface different from thefirst communication interface and based on a second communicationprotocol different from the first communication protocol. Examples ofsuch host devices may include smart phones, desktop computers, laptopcomputers, and other such network enabled computing devices.

Client device 300 may communicate with host device 350 via connection338. Similarly, host device 350 may communicate with network 390 viaconnection 388. In further embodiments, client device 390 may alsocreate a connection with one or more other networks including network390. Different embodiments may implement connection 338, connection 388,or aspects of these connections using one or more communicationprotocols or technologies, including time division multiple access(TDMA), code division multiple access (CDMA), global system for mobilecommunications (GSM), Enhanced Data GSM Environment (EDGE), widebandcode division multiple access (W-CDMA), Long Term Evolution (LTE),LTE-Advanced, WiFi (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11gand/or IEEE 802.11n), Bluetooth, Wi-MAX, voice over Internet Protocol(VoIP), near field communication protocol (NFC), a protocol for email,instant messaging, and/or a short message service (SMS), or any othersuitable communication protocol, including communication protocols notyet developed as of the filing date of this document. A host or clientdevice can include wireless circuitry as part of wireless interfaces,such as wireless interface 312, wireless interface 362, and wirelessinterface 364, that can communicate over several different types ofwireless networks depending on the range required for the communication.For example, a short-range wireless transceiver (e.g., Bluetooth), amedium-range wireless transceiver (e.g., WiFi), and/or a long rangewireless transceiver (e.g., GSM/GPRS, UMTS, CDMA2000 1×/EV-DO andLTE/LTE-Advanced) can be used depending on the type of communication orthe range of the communication.

Connection 338 in particular may, in certain embodiments be implementedas a peer to peer (P2P) wireless connection directly between wirelessinterface 312 and wireless interface 362. In other embodiments,connection 338 may include multiple additional devices andsub-connections, including multiple access points, network routingconnections, and communication servers.

Wireless circuitry may be used in conjunction with wireless interfacessuch as wireless interfaces 312, 362, 364, 462 and 464 to send andreceive information over wireless connections such as connections 338,388, 418, and 488. Any device described herein may additionally includeconventional circuitry such as an antenna system, an RF transceiver, oneor more amplifiers, a tuner, one or more oscillators, a digital signalprocessor, a CODEC chipset, memory, etc. to enable various wirelessconnections as described herein.

Wireless circuitry may be coupled to processors such as processor 302and processor 354 via peripherals interfaces. Voice and data informationreceived by wireless circuitry may be sent to one or more processors viaperipheral interfaces. One or more processors such as processors 302 and352 may be configurable to process various data formats for one or moreapplication programs stored on the memory of a device.

Processors 305 and 352 may run various software components stored inrespective memories 304 and 354 to perform various functions for devices300 and 350. In some embodiments, the software components include anoperating system, a communication module (or set of instructions), andother applications (or sets of instructions). In different embodiments,a processor may be a single integrated circuit, a set of multipleintegrated circuits, or any logic hardware that may be used to implementprocessing functionality.

An operating system executed by a processor can be any suitableoperating system. The operating system can include various procedures,sets of instructions, software components and/or drivers for controllingand managing general system tasks (e.g., memory management, storagedevice control, power management, etc.) and facilitates communicationbetween various hardware and software components.

Audio device 370 further includes microphone 374, speaker 372, andfilters 375. Audio device 370 including speaker 372 and microphone 374may operate as input and output transceiver for a speaker engaging in aphone call using wireless interface 364 and 390. Filters 375 may performprocessing on input and output audio signals to improve data quality inAV data signals created by audio device 370.

Baseband device 359 may be implemented as a separate device or chip fromprocessor 352 as shown in FIG. 3. In alternative embodiments, basebanddevice 359 may be implemented as an integrated device with processor352, as firmware, or as software executed by processor 352. Basebanddevice 359 may operate in conjunction with audio device 370 to enablecalls by managing audio-visual (AV) data from audio device 370communicated to network 390 as part of a call. To implement relay,embodiments may function with baseband device 359 implementing a virtualaudio device that enables applications to access audio functionality andmimic the input received from a microphone. This may enablecommunication of AV data received from network 390 to be treated as ifit was received from audio device 370 as detailed further below.

Memory as described herein, including memory 304 and memory 354, may beany computer-readable medium and may further be any device or mediumthat can store code and/or data for use by one or more processors ordevice components. This may include a memory hierarchy, including cache,main memory and secondary memory. The memory hierarchy can beimplemented using any combination of RAM (e.g., SRAM, DRAM, DDRAM), ROM,FLASH, magnetic and/or optical storage devices, such as disk drives,magnetic tape, CDs (compact disks) and DVDs (digital video discs). Anysuch computer readable medium may reside on or within a single computerproduct (e.g. a hard drive, a CD, or an entire computer system), and maybe present on or within different computer products within a system ornetwork. In some embodiments, peripherals interfaces, one or moreprocessors, and a memory controller can be implemented on a single chipwithin any device described herein. In some other embodiments, they canbe implemented on separate chips.

Any device described herein such as host device 350 and client device300 may also include a power systems for powering the various hardwarecomponents. Power systems can include a power management system, one ormore power sources (e.g., battery, alternating current (AC)), arecharging system, a power failure detection circuit, a power converteror inverter, a power status indicator (e.g., a light emitting diode(LED)) and any other components typically associated with thegeneration, management and distribution of power in mobile devices.

In some embodiments, host devices and/or client devices may includevarious sensors. A camera may be one example of a sensor. Such cameradevices along with other sensors may be considered input devices as partof input device 356. A camera may be used to create AV data for videocalls. In some embodiments, devices may also include various othersensors. Sensors can include accelerometers, compass, gyrometer,pressure sensors, audio sensors, light sensors, barometers, microphones,and the like. Sensors can be used to implement various device decisionmaking and filtering, such as blur compensation for a video image in avideo call, or audio filtering to assist in filtering background noisefrom audio information captured at one or more microphones such asmicrophone 374.

In some embodiments, a client device 300 or a host device 350 caninclude a GPS receiver or other location assistance devices or modules.A device can use a satellite navigation system, such as the GlobalPositioning System (GPS), to obtain position information, timinginformation, altitude, or other navigation information. Duringoperation, the GPS unit can receive signals from GPS satellites orbitingthe Earth. The GPS unit analyzes the signals to make a transit time anddistance estimation. The GPS unit can determine the current position(current location) of the mobile device. Based on these estimations, themobile device can determine a location fix, altitude, and/or currentspeed. A location fix can be geographical coordinates such aslatitudinal and longitudinal information. Other non-GPS location/motionmodules may can assist in determining the current position (e.g.,coordinates or other geographic location identifier) and motion of aclient device 300 or host device 350. In addition to GPS, cellularnetwork positioning based on “cell IDs,” and WiFi positioning technologybased on a WiFi networks may be used for location assistance. In someembodiments, location/motion modules may use sensor information with theabove systems to determine the current position of the mobile device andto assist a user in finding the device.

Processor 302 and processor 352 may execute one or more applications ontheir respective devices, including without limitation, a browser,address book, contact list, email, instant messaging, word processing,keyboard emulation, widgets, JAVA-enabled applications, encryption,digital rights management, voice recognition, voice replication, a musicplayer (which plays back recorded music stored in one or more files,such as MP3 or AAC files), etc. The one or more applications can alsoinclude a specific application for managing communications or formanaging calls with relay functionality.

There may be other modules or sets of instructions such as a graphicsmodule, a time module, etc. For example, the graphics module can includevarious conventional software components for rendering, animating anddisplaying graphical objects (including without limitation text, webpages, icons, digital images, animations and the like) on a displaysurface. In another example, a timer module can be a software timer. Thetimer module can also be implemented in hardware. The time module canmaintain various timers for any number of events.

I/O subsystems including various input devices can be coupled to adisplay system such as display 308 or display 358. These displays may betouch-sensitive and may therefore operate as displays and input devices.A touch sensitive display of client device 300 may thus operate both asinput device 306 and display 308. A touch sensitive display of host 350may operate as display 358 and input device 356. The display may presentvisual output to the user in as a graphic user interface (UI). Thevisual output can include text, graphics, video, and any combinationthereof. Some or all of the visual output can correspond touser-interface objects. A display can use LED (light emitting diode),LCD (liquid crystal display) technology, or LPD (light emitting polymerdisplay) technology, although other display technologies can be used inother embodiments.

In some embodiments, an I/O subsystem can include a display and userinput devices such as a keyboard, mouse, and/or trackpad as part ofinput device(s) 306 or input device(s) 356. In some embodiments, I/Osubsystem can include a touch-sensitive display. A touch-sensitivedisplay can also accept input from the user based on haptic and/ortactile contact. In some embodiments, a touch-sensitive display forms atouch-sensitive surface that accepts user input. The touch-sensitivedisplay/surface (along with any associated modules and/or sets ofinstructions) detects contact (and any movement or release of thecontact) on the touch-sensitive display and converts the detectedcontact into interaction with user-interface objects, such as one ormore soft keys, that are displayed on the touch screen when the contactoccurs. In some embodiments, a point of contact between thetouch-sensitive display and the user corresponds to one or more digitsof the user. The user can make contact with the touch-sensitive displayusing any suitable object or appendage, such as a stylus, pen, finger,and so forth. A touch-sensitive display surface can detect contact andany movement or release thereof using any suitable touch sensitivitytechnologies, including capacitive, resistive, infrared, and surfaceacoustic wave technologies, as well as other proximity sensor arrays orother elements for determining one or more points of contact with thetouch-sensitive display.

Further, the input devices of a particular client device or host devicecan be coupled to one or more other physical control devices (notshown), such as pushbuttons, keys, switches, rocker buttons, dials,slider switches, sticks, LEDs, etc., for controlling or performingvarious functions, such as power control, speaker volume control, ringtone loudness, keyboard input, scrolling, hold, menu, screen lock,clearing and ending communications and the like. In some embodiments, inaddition to the touch screen, devices may include a touchpad separatefrom the screen for activating or deactivating particular functions. Insome embodiments, the touchpad is a touch-sensitive area of the devicethat, unlike the touch screen, does not display visual output. Thetouchpad can be a touch-sensitive surface that is separate from thetouch-sensitive display or an extension of the touch-sensitive surfaceformed by the touch-sensitive display.

In some embodiments, some or all of the operations described herein canbe performed using an application executing on the user's mobile device.Circuits, logic modules, processors, and/or other components may beconfigured to perform various operations described herein. Those skilledin the art will appreciate that, depending on implementation, suchconfiguration can be accomplished through design, setup,interconnection, and/or programming of the particular components andthat, again depending on implementation, a configured component might ormight not be reconfigurable for a different operation. For example, aprogrammable processor can be configured by providing suitableexecutable code; a dedicated logic circuit can be configured by suitablyconnecting logic gates and other circuit elements; and so on.

Computer programs incorporating various features of the presentinvention may be encoded on various computer readable storage media;suitable media include magnetic disk or tape, optical storage media suchas compact disk (CD) or DVD (digital versatile disk), flash memory, andthe like. Computer readable storage media encoded with the program codemay be packaged with a compatible device or provided separately fromother devices. In addition program code may be encoded and transmittedvia wired optical, and/or wireless networks conforming to a variety ofprotocols, including the Internet, thereby allowing distribution, e.g.,via Internet download.

In addition to client devices such as client devices 300 and 400A-Z, andhost devices such as host devices 350 and 450 detailed above, aspects ofnetworks such as network 390 and network connections such as networkconnections 338, 388, 418, and 488 may also be implemented using variousdevices and device components. For example, while in certain embodimentsconnection 338 may simply be a P2P wireless connection directly betweenwireless interface 312 and wireless interface 362, in other embodiments,a communications network such as comm. network 140 may be used toimplement connection 338. Such a connection may include one or moreaccess points, routers, server computers, storage devices, or otherdevices to implement network connection 338. Each of these devices mayinclude a processor, memory, and input/output structure as detailedabove. Similarly, other network connections may involve cell towers,switching networks, packet processing devices, in addition to the servercomputers and other network infrastructure detailed above, each of whichmay include processing, memory, and network interface components.

Although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

FIG. 4 includes client devices 400A-Z, host device 450, and network 490.Client devices 400A-Z may each be similar to client device 300, andnetwork 490 may be similar to network 390 as described in detail abovewith respect to FIG. 3. Host device 450 host device 450 may be similarto host device 350. While host device 350 shows one embodiment ofcomponents that may make up a portion of a host device, host device 450is shown as including elements that may be implemented in whole or inpart as software, firmware, and/or hardware. For software or firmwareembodiments, these elements may be implemented in whole or in part aprocessor and memory such as processor 352 and memory 354. Theseelements of FIG. 4 may also be implemented in conjunction with otherhardware components shown in host device 350, or using other componentsor groups of components which are not specifically illustrated.

Host device 450 is shown as including user interface 451, relayapplication 452, IDS 453, signaling 454, A/V Conference 456, basebandI/O 474, baseband audio unit 472, audio unit 470, and wirelessinterfaces 462 and 464.

User interface 451 may refer to a display output, a set of optionsassociated with user input actions, or a combination of display outputsand associated user input options. A user interface may thus beimplemented using a combination of a display such as display 358, inputdevice(s) such as input device 356, and associated logic andpresentation as implemented using a processor and memory such asprocessor 352 and memory 354. For example, user interface 451 maycomprise a display on a touch screen interface with an answer interfacepresented on the display including an answer button and a calleridentification window. Other buttons may also be present including anignore button and additional information related to the counterparty onthe other side of the call. A relay button may also be present in suchan interface to initiate a relay of the call to client devices 400A-Z.

User interface 451 may have multiple forms depending on a call state,such as an incoming call interface, a call answered interface, a callhold interface or other such state dependent forms. Each form may beassociated with a different display that is output to the user, andwhich identifies different user selection options for each form of userinterface 451. As a state of the device changes, either in response to auser selection at user interface 451, in response to an automatic changesuch as a call timeout, or in response to a counterparty communicationreceived at a wireless interface, the form of the user interface 451 maychange. For example, an incoming call interface may change to a callanswered interface in response to receipt of an answer call inputreceived at user interface 451. Alternatively, an incoming callinterface may change to a default interface if a counterparty terminatesthe incoming call prior to any selection being made at user interface451.

Relay application 452 may be any application that implements relayfunctionality as described herein. In certain embodiments, relayapplication 452 may essentially act as an interface for otherapplications operating on host device 450 to implement relayfunctionality in conjunction with other modules. In other embodiments,relay application may be any type of application integrated with relayfunctionality, including a call application, a game application, amulti-purpose communication application, or any other such applicationthat may operate using a processor of host device 450. Relay application452 may interact with identity services (IDS) 453, signaling 454,baseband audio unit 472, and user interface 451 to manage implementationof relay functionality. In other embodiments, relay application 452 mayintegrate functionality from other modules, or may communication withother modules any fashion to implement relay functionality as describedherein.

IDS 453 may be a module that operates on host device 450 to implementidentity services as part of relay functionality. This may be done inconjunction with additional IDS services such as identity managementinfrastructure 105 of FIG. 1 and IDS 205 of FIG. 2. In certainembodiments, IDS 453 may be considered an interface for interacting withan identity management server via a wireless interface. IDS 453 mayinclude an address of an identity management server and an identityassociated with host device 450.

As part of implementing relay functionality, IDS 453 may communicatewith an identity management server to identify client devices 400A-Z asbeing associated with the same identity or a related identity that isalso associated with host device 450. IDS 453 may also manageconnections between host device 450 and client devices 400A-Z once theassociation of the devices with an identity group is confirmed. Incertain embodiments, IDS 453 may store a host identifier for host device450 that is received from an identity management server. IDS 453 mayadditionally store client identifiers for client devices that have beenidentified as part of an identity group or that shares an accountidentity. Each of these identifiers may be unique identifiers that allowthe devices to distinguish one device from another during call relayoperations.

Signaling 454 and AV conference 456 may each comprise, at least in part,communication ports opened by IDS 453 as part of communication and relayof signals between host device 450 and one or more of client devices400A-Z. For example, if an input at user interface 451 causes relayapplication 452 to request communication with client devices sharing anidentity with host device 450, IDS 453 may communicate with an identitymanagement server to identity client devices 400A-Z as sharing anidentity with host device 450. IDS 453 may then use the information fromthe identity management server to open a channel for use in overheadcommunication with each of client device 400A-Z as part of signaling454.

Such a channel may use, in certain embodiments, TCP for communicationsand may involve opening an internet protocol (IP) socket for use incommunications between client device 400A and host device 450. In otherembodiments, any other such communications protocols may be used.Similarly, if relay of an audiovisual (AV) communication is initiatedwith a particular client device, IDS 453 may manage opening an IP socketthat is particularly for the AV communication as part of AV conference456. This may be a separate channel than the channel used by signaling454, even though the communication is between the same devices.

In certain embodiments, as part of relaying a call, AV conference 456may include adaptive channel management based on AV communication signalquality. For example, AV conference 456 may function as part of a systemto identify dropped packets or channel degradation, and may adjustencoding or aspects of a signal to manage call performance during arelay process. As part of such adaptive channel management, AVconference 456 may work in conjunction with baseband I/O 474 to manageencoding of AV data for communication to client devices. Baseband I/O474 may, in other embodiments, manage encoding of AV data from anysource for communication to and from client devices and/orcounterparties to a call across network 490.

Additionally, in certain embodiments, signaling 454 and/or AV conference456 may work in conjunction with IDS 453 to assign various identifiersas part of call relay. For example, in certain embodiments, IDS mayprovide any client device or host device with a unique identifier to beused during a call. In additional embodiments, multiple connections maybe opened between a single host device and a single client device, andIDS 453 operating as part of a larger identity services system mayprovide each connection with a unique identifier. Thus, if one IP socketis opened for signaling and control communications between a host and afirst client device, and a second IP socket is opened for AVcommunications, each socket or a channel associated with each socket maybe assigned a unique identifier by the IDS 453.

Audio unit 470 may be similar to audio unit 370, and may includemicrophone, speaker, filtering, and other AV communication components.In certain embodiments, this may include communications with a wirelessheadset or with a wired set of speakers. Other embodiments may includeany such audio device components. Baseband input/output (I/O) 474 mayrefer to hardware, firmware, or software for managing AV communicationsas part of host device 450. In a cell phone communication between hostdevice 450 and a counterparty phone via network 490, baseband I/O 474may manage microphone input signals and speaker output signals fromaudio unit 470.

In addition to managing signals from an audio unit 470, baseband I/O 474may also manage AV communication signals from baseband audio unit 472.Baseband audio unit 472 may operate, in certain embodiments, as avirtual audio unit which may provide various applications such as relayapplication 452 with access to baseband I/O 474. As part of relayfunctionality, this may enable AV communications from network 490 thatwould be sent to audio unit 470 for output via a speaker to instead besent to baseband audio unit 472 for relay to another device via AVconference 456. AV conference may thus receive from a client device, anuplink conference audio stream, and input the uplink conference audiostream to the baseband I/O 474. Baseband I/O may output an uplink audiostream, and this stream may be communicated to the network 490 as thehost device 450 receives a downlink stream of audio data from thenetwork 490 as part of the call.

In certain embodiments, baseband audio unit 472 may be structured toreceive the same signal that audio unit 470 would receive. Instead ofoutputting the signal on a microphone, however, baseband audio unittransfers the signal. This may be considered as providing a resultsimilar to holding the output of a first phone speaker next to the inputmicrophone of a second phone in order to relay a signal to a clientdevice such as client device 400A. Baseband audio unit 472 mayadditionally receive AV communications from AV conference 456 andbaseband I/O 474 via wireless interface 462 to relay the AVcommunication to a call counterparty using wireless interface 464 andnetwork 490. Baseband audio unit 472 may thus, in certain embodiments,enable bypassing the audio unit 470 of host device 450 in order to relayAV communications between network 490 and a client device.

While FIG. 4 illustrates one embodiment of a system of modules andconnections between modules, alternate embodiments may combine or enablecommunications between such modules in any possible way. For example, incertain embodiments, relay application 452 may encompass IDS 453 andbaseband audio unit 472. In alternate embodiments, AV communicationsfrom wireless interface 464 may be communicated to relay application 452by additional communications modules. In further embodiments, basebandI/O 474, baseband audio unit 472, and audio unit 470 may be combined orintegrated in a variety of ways. In still further embodiments, multipleapplications in addition to relay application 452 may access basebandaudio unit 472 to enable the applications access to a virtual audio unitand associated functionality.

III. Relaying a Call to a Client

A. Sequence Diagram

FIG. 5 describes one method of making a call using relay. The chart ofFIG. 5 is illustrated as using a first client device 300, a host device350, and a network 390 from FIG. 3. In alternate embodiments, any clientdevice, host device, and network may be used. The chart of FIG. 5additionally shows one potential sequence, with the sequence progressingfrom top to bottom. Alternative embodiments may be implemented with theorder altered in any way possible given the specific dependencies ofsignals and inputs.

In initial blocks 502 and 504 include device registration with anidentity service. This enables client device 300 and host device 350 tobe associated as part of an identity or a group of identities. This alsoenables each device to be given a unique identifier by an identitymanagement system, so that each particular device may be easilyidentified during relay communications.

As another initial block, 506 includes receipt of call relay settings athost device 350. This may include a selection for call relay to be usedfor all incoming calls, particular groups of incoming calls, callsduring certain times or trigger periods, or any other such settings.Block 506 may also include settings for which client devices are toreceive relay messages. This may include any combination of selectionsfor particular client devices that are associated with an identity oraccount shared by host device 350.

In block 508, network 390 carries a call from a counterparty, andcommunicates the call to host device 350. In block 510, host device 350receives the incoming call request, and based on the settings from block506, relays notice of the call to one or more client devices. This relaycommunication may include an incoming phone number, a host device ID, auser identity or account identity associated with both the client deviceand the host device, and other IDs associated with the communicationsuch as a video ID or other metadata associated with the call. Thisrelay process may be similar to the process described for FIG. 4, wherean IDS module of host device 350 accesses an IDS server to identify aplurality of clients including client device 300. The host device 350may then communicate notice of the incoming call to all of theassociated client devices. As part of a response to receipt of theincoming call, host device 350 may also update a user interface toindicate that a call is incoming, and that notice has been relayed tothe client device or devices. In certain embodiments, relay of theincoming call request may use a signaling module such as signaling 454of FIG. 4.

In block 512, client device 300 receives notice of the call by a relayedcall request. Client device 300 may then display a user interfaceindicating that a call is incoming at host device 350. When an input isreceived via the user interface to accept the call, a call acceptancemay be sent to host device 350. Such a call acceptance may include aunique identifier assigned by an identity services system as discussedabove. It may also include the user identity associated with both thehost device and the client device, a start time, any number of systemstate flags, and any other such metadata. In certain embodiments, a userinterface may be updated upon receipt of the accept call input, while inother embodiments, the user interface update may wait for connection ofthe call, or the user interface may be updated multiple times at clientdevice 300 through this part of the process.

In block 514, host device 350 receives notice that an accept call inputwas received at client device 300. If multiple client devices wereinformed of the incoming call, host device 350 may then communicate anupdate to the informed client devices other than client device 300 thatthe call has been accepted. Host device 350 may also implement processesto manage conflicting call accept inputs from multiple devices.

In blocks 516 and 518, an AV conference is opened to enable relay of AVcommunications from host device 350 to client device 300. In block 520,an audio unit of host device 350 is muted, and in blocks 522 and 524, anetwork connection for the incoming call is completed. This process may,in certain embodiments, be the same process for connecting a call toenable a user of host device 350 to speak as part of the call. The audiounit 520 is muted during this process because, although the call isconnected, the relay is taking the place of the audio unit functionalitydue to the relay process. Relay operation without muting audio unit 520may, in some embodiments, result in automatic engagement of a speakerand microphone at host device 350 which may interfere with an end to endcall between client device 300 and network 390 created by relay of thecall from host device 350 to client device 300.

This end to end call is established at blocks 526, 528, and 530 based onthe processes above. As part of these blocks, AV communications are sentand received from a call counterparty over network 390. Similarly,client device 300 sends and receives AV information as part of the AVcommunication. Host device 350 relays the information between client 300and network 390 as part of block 528. This call relay continues until adisconnect input is received at block 532 or 536. Such a disconnectinput may be received in a variety of ways.

In certain embodiments, the call may continue after the relay isdisconnected if an input to discontinue the relay is received via a userinterface of either client device 300 or host device 350. Following suchan input, the call may continue between host device 350 and network 390.A disconnect input may also be received via a user interface of eitherclient device 300 or host device 350. In either case, an AV conferenceusing a connection between client device 300 and host device 350 will becleaned up after termination of the relay, and in block 534, a userinterface of the client device 300 will return to a default state.Similarly, after the call is disconnected in blocks 538 and 540, anyactions taken in connection with the call may be cleaned up at hostdevice 350, and a host device user interface may return to a defaultstate.

B. Host Device

FIG. 6 describes a method 600 of relaying a call between a client deviceand a host device according to embodiments of the present invention.FIG. 6 describes a method performed at a host device. The host devicemay be similar to host device 350, host device 450, or any other suchhost device. As indicated, in embodiment of FIG. 6 involves a hostdevice with a first wireless interface for communicating to a firstnetwork, a second wireless interface for communicating to the firstclient device, a memory, a display, and a processor coupled to the firstand second wireless interfaces, to the memory, and to the output display(e.g., a touch screen display).

At block 602, the first wireless interface of the host device receives avoice invite communication from a first network as part of the call. Thevoice invite communication may be for an audio call, a videocommunication, or a call including both audio and video. As used herein,AV communication, voice communication, or call refers to a communicationthat may include either audio or video only, or both. Examples of thefirst wireless interface include radio network interfaces, such as 3G,4G, and LTE. A voice invite communication is a signal notifying the hostdevice that there is an incoming call. Such a communication can be whena phone receives a phone call from another phone via a radio network.The voice invite communication may additionally include metadata such asan identifier associated with a device or counterparty that initiatedthe call.

At block 604, the output display of the host device displays acall-received interface on the output display in response to receivingthe voice invite communication. Such an interface may include calldetails, display of information associated with physical buttons, orcreation of virtual buttons on a touchscreen. Call details can includecaller ID information, which may include a name from a contact list.Examples of such an interface can include one or more of the followingbuttons: an answer button, a hold button, a forward button, and adecline button.

At block 606, a first client device is identified by accessing a list ofone or more client devices associated with the user. The list includes adevice identifier for each client device. The host device can store thelist in the memory of the host device, e.g., in flash memory or RAM. Thehost device can retrieve the list from a server, and then temporarilystore the list in memory. Thus, this list may be accessed bycommunicating with an identity management server after receiving thevoice invite communication.

The client device identifiers can be used to ensure that the clientdevice is an approved device, which may be determined when the voiceinvite communication is sent. The client identifier may also be used toidentify the communication as being part of the first connection, so thehost device knows how to process the communication, e.g., how to respondto the first network.

More client devices than the first client can be identified. Similarprocedures can be performed for the other client devices. For example,at least some of steps 608-622 can be performed in parallel for multipleclient devices. Depending on which client device responds to with ananswer communication, the call can be forwarded to that client device.

At block 608, the second wireless interface of the host device relaysthe voice invite communication to the first client device using a firstidentifier associated with the first client device. As recited here,relaying the voice invite communication may include sending a copy orpart of a copy of a communication received at the host device. This mayalso include creating a new notice in response to receipt of the voiceinvite communication, and sending the notice. Either wireless interfacedevice may be configured to communicate any time of wireless signaldescribed herein.

Additionally, in certain embodiments, the first wireless interface andthe second wireless interface may be different physical devices fordifferent wireless communication types. For example, the first wirelessinterface may include a wide area network antenna, and the secondwireless interface may include a local area network or near fieldantenna. In other embodiments, the first wireless interface may be thesame interface as the second wireless interface. The voice invitecommunication may include the same information as was sent by the firstnetwork or can have a different form that includes more or lessinformation that was sent by the first network.

At block 610, the second wireless interface receives a client-answercommunication from the first client device. In alternate embodiments,the client-answer communication may be received via a differentinterface other than the interface that was used to communicate thevoice invite communication. The client-answer communication can be anysignal that indicates that the user has accepted the call on the firstclient device.

If the second wireless interface receives multiple client-answercommunications from different client devices, then the host device canselect one of the client devices based on one or more criteria. Forexample, a priority list can be set by the user at the host device for apreferred order of client device, where the client device with thehighest priority that provides an answer can be selected. A particularphone may have highest priority, e.g., when that phone is a default formaking video calls (e.g., a default set by the user). In one embodiment,only the top host device is provided to the user. In such a case, theuser effectively selects the phone by requesting a call to be made.

At block 612, the host device establishes a first connection for thecall using the first wireless interface. The first connection can be asame connection as used in block 602 for receiving the voice invitecommunication. The first connection can also be a different connectionthan used in block 602. In one implementation, the first connection canproceed as if a user had answered the call on the host device.

At block 614, the first wireless interface sends a host-answercommunication to the network using the first connection. Block 614 mayinvolve, in part, receiving, from the network via the first wirelessinterface, a call-connected communication. This may involve multiplerounds of communication and acknowledgement transmissions as part of ahandshaking process to establish the connection. The first connectionmay then provide communication of voice data and/or video data andpossibly other metadata between the host device and the network. Thehost-answer communication can be any of the signals from the host devicethat indicates an answer of the call to the first network.

At block 616, the host device establishes a second connection with thefirst client device using the second wireless interface and the firstidentifier. The second connection provides communication of voice and/orvideo data between the first client device and the host device inresponse to the client answer communication. The second connection canbe the same or a different connection as used to convey theclient-answer communication. The first identifier can be used to ensurethat the second connection is made to the client device that answeredthe invite communication.

At block 618, the host device receives a call-connected communicationfrom the network via the first wireless interface. The call-connectedcommunication can occur before or after the establishing of the secondconnection with the first client device.

At block 620, the host device updates a status of the call to activebased on the call-connected communication. A call status can bemaintained in working memory, e.g., so the host device can continue toforward data between the first client device and the first network.

At block 622, the second wireless interface relays the call to the firstclient device. The host device may then continue relaying the AVinformation until either the relay is terminated and the host device ismade an endpoint of the call, or until the call is terminated and theconnections associated with the call are closed and cleaned-up.

C. Client Device

FIG. 7 describes a method 700 related to method 600, but correspondinglyperformed by a client device which acts as the end point of acommunication being relayed by an associated host device. The firstclient device performing the method of FIG. 7 may be any client devicedescribed herein, and includes a wireless interface, an output display,and a processor coupled to the wireless interface and the outputdisplay.

At block 702, a first client device receives, from a host device via thewireless interface of the client device, a voice-invite communication aspart of the call. The voice-invite communication includes a hostidentifier that identifies the host device. Examples of the voice-invitecommunication are provided above. The first client device can use thehost identifier for establishing a connection with the host deviceshould the call be answered at the first client device. In anotherexample, the first client device can use the host identifier to ensurethat the host device is associated with the same user as the firstclient device. For example, the first client device can check the hostidentifier against a list of known devices of the user, where the listcan be stored at the client device or accessed by the first clientdevice from a server (e.g., a server of identity managementinfrastructure 105). The host identifier can be used to facilitate anycommunications between the first client device and the host device.

At block 704, a call-received interface can be provided by the firstclient device in response to receiving the voice invite communication.The call-received interface may be visual by display on the outputdisplay or by voice, e.g., a voice command interface. Such an interfacemay include call details, display of information associated withphysical buttons, or creation of virtual buttons on a touchscreen. Calldetails can include caller ID information, which may include a name froma contact list. Examples of such an interface can include one or more ofthe following buttons: an answer button, a hold button, and a declinebutton. An audible ring can be output in response to the receiving thevoice invite communication.

At block 706, the call-received interface receives an input to answerthe call. The input may be provided via the output display (e.g., via atouch selection of a button) or via voice command. For example, a usercan select an answer button on the output display, or the user can say“answer” when prompted by the call-received interface.

At block 708, the first client device establishes a first connection forthe call. The first connection provides communication of audio betweenthe client device and the host device. This connection may, for examplebe performed using the wireless interface and an AV conferenceimplemented partially on the host device and partially on the clientdevice via the channel and wireless interfaces coupling the devices. Thefirst connection can be a same connection used to receive thevoice-invite communication. In another embodiment, the first connectioncan be newly generated and be separate from a connection used to receivethe voice-invite communication.

At block 710, the first client device communicates a call-answercommunication to the host device in response to the input to answer thecall. The call-answer communication can comprise a client identifierassociated with the first client device. The host device can use theclient identifier, e.g., so to determine which client device to forwardthe call. Such a determination by the host device can include a prioritylist, as described above. The client identifier can also be used toensure that the client device is an approved device, which may bedetermined when the voice invite communication is sent. The clientidentifier may also be used to identify the communication as being partof the first connection, so the host device knows how to process thecommunication, e.g., how to respond to the first network.

At block 712, a call status is updated to active. For example, the firstclient device can store a call status in working memory to identifywhether to continue with the first connection to exchange data with thehost device. Following this, the first client may act as an endpoint ofthe call, receiving AV information from a counterparty to the call via anetwork and the host device and communicating AV information back alongthe same path until the relay is terminated.

At a later time, the user can select to end the call at the first clientdevice. Such an input can be received at an end call button on theoutput display. The first client device can send the end response to thehost device so that the host device can close the connection to thenetwork. The first client device can close the first connection with thehost device in response to the end input.

IV. Initiating a Call at a Client

A. Sequence Diagram

FIG. 8 describes a method where a call may be initiated by a clientdevice of a user to be made via a host device of the user according toembodiments of the present invention. Further, a handoff (transfer) maybe performed to move an endpoint of a call from the initiating clientdevice to the relay host device. As mentioned above, such a relay mayalso be performed when a host device is receiving an incoming call froma counterparty as described in FIG. 5.

In blocks 802 and 804, first client device 300 and host device 380 mayregister with an identity management system. This may provide thedevices with unique identifiers and associate the devices with anidentity or account. As part of such a registration and association, thedevices may store unique identifiers for devices sharing an identitylocally, which enables the host device to locate client devices withoutthe need to access an IDS server prior to initiating a relay. In variousembodiments, registration may be renewed or updated periodically orbased on various triggers such as system updates or alerts pushed fromthe identity system.

In block 806, call settings may be initialized. For example, if anidentity or account is associated with multiple host devices, thesettings may identify which host devices may be used with the firstclient device 300, or a priority list for which host devices the firstclient device 300 should attempt to use first to initiate a call relay.In block 808, call relay settings may be set at host device 380. Thismay include settings allowing outgoing calls from particular clientdevices. This may also include timeout settings, which may disconnect arelay after a connection to a first client device 300 or network 390 issilent for a period of time determined by the call relay settings.

In block 810, a default user interface is initially displayed with aninitiate call input option. A place call input is received at the userinterface, and a communicate call message is sent from first clientdevice 300 to host device 380. In block 812, host device 380 receivesthe call request from the first client device 300, and updates a userinterface. In Blocks 814 and 816, an AV conference is opened with achannel for the relay portion of the call. In block 818, the audio unitof host device 380 is muted, and in blocks 820 and 822, the networkconnection for the call is established. In various embodiments,communications as part of establishing a connection for the call such asa ring tone or a busy signal may be relayed back to first client device300 so that sound associated with these communications may be played atfirst client device 300. While the opening of the AV conference andmuting of the audio is shown prior to establishing the networkconnection to network 390 for the call, in certain embodiments, the AVconference may be established after the call is connected.

In blocks 824, 826, and 828, AV information as part of the call isrelayed between network 390 and first client device 300 as part of thecall. Following this, as shown by blocks 830 through 840, the endpointof the call may be transferred from the first client device to the hostdevice. In block 830, a call handoff input (transfer request) isreceived at first client device 300. The AV conference is then closed inblocks 830 and 834, and the user interface at the first client device isupdated at block 832.

In block 836, the audio unit is unmuted, and the host device becomes anendpoint of the call. In certain embodiments, this unmuting process maybe performed in a variety of different ways. For example, referring toFIG. 4, during call relay, AV information coming from the network 390may be sent to baseband audio unit 472 (either directly, via relayapplication 452, or via some other path), then to baseband I/O 474, thento AV conference 456 where it is transmitted to a client device viawireless interface 462. AV information coming from the client device mayfollow the reverse path to network 390. Audio unit 470 is not part ofthe communication and may be blocked.

When a handoff (transfer) input is received, either at the client deviceor the host device involved with the relay, the AV information fromnetwork 390 is redirected to the audio unit 470, and creates an AVoutput at a microphone and possibly a display associated with audio unit470. In certain embodiments, this will bypass baseband audio unit 472,and send the AV information to audio unit 470 via baseband I/O 474.Similarly, AV input from a microphone and/or camera associated withaudio unit 470 may then be sent to network 490 via baseband I/O 474,bypassing baseband audio unit 472.

In various other alternative embodiments, any such changes in moduleconnections may be made to transfer the AV information from a relay modethat passes the AV information received at a host device between aclient and a network, to a local mode where the host device is anendpoint that outputs AV information at an audio unit and/or a displayand that captures AV information via a microphone and/or a camera topass to a call counterparty via a network.

In blocks 836 and 840, AV information is communicated between the hostdevice and the network as part of the call until the call is terminatedor transferred from the host to a client device, which may be either thefirst client device or another client device, is performed.

B. Client Device

FIG. 9 describes a method 900 for initiating a relayed call using aclient device according to embodiments of the present invention. Themethod of FIG. 9 may, in certain embodiments, be performed by a clientdevice that includes a first wireless interface, an output display, anda processor coupled to the wireless interface and the output display.

At block 902, the client device provides a user interface for requestingan outgoing call at the client device. As part of this provision, theclient device may initially communicate with an identity service, andreceive contact details for one or more host devices. The device mayalso send an initial query to the host to insure that the host ispowered on, connected through an acceptable network, and available for acall. If the host device is powered off, inaccessible via a wirelessconnection, or already engaged in a call, the user interface may displaythis information. The client device may periodically poll the hostdevice for a status update to see if it is available for call relay, andmay update the user interface when the host device becomes available.

At block 904, a selection of a first host device is received by the userinterface of the client device. The selection identifies the first hostdevice for use in implementing the outgoing call. The selection can beprovided in various ways. For example, the client device can provide alist of possible host devices to use. The list can be determined fromuser input of known devices of the user or previous communications. Thecurrent list provided to the user can be dynamically confirmed beforeproviding to the user, e.g., the client device can attempt tocommunicate with the host devices.

Host device(s) to which a connection could not be made may not beprovided to the user to select, as such host device(s) can be consideredas not available. Further, the host devices can be presented in a rankedorder according to criteria. Example criteria for the order can be speedof connection from the host device to a network for making the call,current connection status of the host device to the network, batterystatus of the host device, and current battery usage. The order may alsobe set by the user. If no host devices are available, an error signalcan be sent to the user, informing the user that initiating a call via ahost device of the user is not available. The error signal can beprovided before providing a list of host devices, or after providing alist (e.g., when the host devices are not confirmed before providing thelist).

At block 906, a first host identifier that identifies the first hostdevice is obtained. In one embodiment, the first host identifier isstored in the memory of the client device. The memory may be part of anIDS application or service operating on the client device, or may bestored as party of a relay application or any other application, module,or program used to implement the call relay. In another embodiment, thefirst host identifier may be obtained from a server. The host identifiercan be used in later steps of method 900.

At block 908, the client device sends a call request that includes arecipient number to the host device via the first wireless interface ofthe client device. The call request requests that the host deviceinitiate an outgoing call to the recipient number. The outgoing call ismade using a second interface protocol that is different than one ormore first wireless interface protocols of the first wireless interface.In one embodiment, the client device can use the host identifier toestablish a connection between the client device and the host device.The call request can be sent over the connection, the connection havinga connection identifier.

The recipient number may be provided via any suitable user interface.For example, the user can type a 10 digit phone number, or anyinternational phone number. The recipient number can also be anysuitable identifier, e.g., an e-mail or instant messaging address thatcan accept a call.

At block 910, the user interface may update a status of the outgoingcall to dialing. The status of dialing may be provided in various ways,which can show progress or an action being taken. A current status canbe stored in working memory of the client device.

At block 912, the client device may then receive a call-answeredcommunication from the host device after the host device is incommunication with a recipient device. The recipient device correspondsto the recipient number. The client device may then use an AV conferenceconnection to send and receive AV information until the relay or thecall is terminated. The AV conference connection can be a persistentconnection between the client device and the host device for sendingdata for the call. The data can be audio and/or video data.

In one embodiment, the call-answered communication can includes the hostidentifier. The host identifier can be used to associate the callanswered communication with the outgoing call. In this manner, theclient device can know that the call-answered communication pertains tothe status of the outgoing call.

At block 914, the status of the outgoing call is updated to active. Theactive status can indicate to the client device that data for the phonecall should be continued to be routed to the host device for sending tothe recipient device. At a later time, the user can provide an endrequest to end the call. At that time, the end request can be sent tothe host device to end the call. And, the connection between the clientdevice and the host device can be closed.

In one embodiment, as part of ongoing communication with the host devicefor the call, the client device can receive call data messages from thehost device. The call data messages can include a connection identifierand call data. The call data can be from the network and the connectionidentifier can correspond to the connection between the client deviceand the host device. Using the connection identifier, the call data canbe routed to a speaker or display of the client device. The connectionidentifier can include or be based on the host identifier, e.g., alongwith an identification that the connection is for a call. Since only onecall would typically occur between the client device and the hostdevice, the host identifier and a flag for a connection type canuniquely identify the connection.

C. Host Device

FIG. 10 describes a method 1000 for relaying, by a host, a callinitiated by a client device according to embodiments of the presentinvention. In one embodiment of FIG. 10, the host device implementingthe method may be made up of at least a first wireless interface forcommunicating to a first network, a second wireless interface forcommunicating to the first client device, a memory, an output display,and a processor coupled to the first and second wireless interfaces, tothe memory, and to the output display.

At block 1002, the host device sends a signal to the client deviceindicating the host device is in communication with the client device.For example, the signal can be in response to the client deviceconfirming which host devices are available, which can occur beforeproviding a list of host devices to the user. The signal can alsoinclude information about network connectivity of the host device, e.g.,whether the host device has a sufficient signal for making an outgoingcall.

At block 1004, a call-initiate request is received at the host device.The call-initiate request can include a client identifier of the firstclient device and a recipient number of a recipient device. The clientidentifier can uniquely identify the client device, e.g., among otherclient devices of the user. The recipient number can be received in anysuitable form, e.g., as mentioned herein.

At block 1006, the host device establishes a first connection for thecall in response to the call-initiate request. The first connectionprovides communication between the host device and the network. Thefirst connection can be created in a similar manner as if the user hadinitiated the call at the host device, e.g., by dialing the recipientnumber at a user interface of the host device. A second connectionbetween the first client device and the host device via the secondwireless interface using the client identifier may also be established.

At block 1008, the first wireless interface sends a host-callcommunication to the network using the first connection. The host-callcommunication can be implemented as a normal call by the host devicewould occur. The host device can track the call (e.g., with a callidentifier) so that the host device knows to relay the call to theclient device.

At block 1010, the first wireless interface receives a call-connectedcommunication from the network. The call-connected communication cansignal to the host device that data for the call can be communicatedbetween the host device and the network. The host device can saveconnection information related to the first connection in preparationfor a connection to the client device, and relaying data between theclient device and the network.

At block 1012, the host device can update a status of the call to activebased on the call-connected communication. In addition to updating thestatus of the call, the host device can institute routing rules for datareceived for the first connection. The first connection can be aconnection identifier associated with it. Data received from the networkcan include the connection identifier, thereby informing the host devicethat the data is associated with the call.

At block 1014, the host device can relay the call-connectedcommunication to the first client device via the second wirelessinterface using the second connection. In one embodiment, the hostdevice can receive an acknowledgement from the client device as toreceiving the call-connected communication, and that the client deviceis ready to send and receive data for the call.

The host device may then continue operating to relay call informationbetween the client device and the network until the relay operation isended. The data can be relayed using an association between the firstconnection and the second connection. For example, data received overthe second connection can have a second connection identifieridentifying the second connection. The client identifier could functionat least in part to provide the second connection identifier. The hostdevice can then use routing rules to identify the first connection asbeing associated with the data, and then forward the data to thenetwork. Data received from the network can be routed to the clientdevice in a similar manner.

V. Protocol Stack

FIG. 11 shows a protocol stack 1100 for communicating data according toembodiments of the present invention. In various embodiments, theprotocol stack 800 may be implemented as part of any host or clientdevice described herein, or as part of any other device that may be usedwith various embodiments. In certain embodiments, the protocol stack1100 may be used to manage retransmission priority either in whole or inpart. Various modules in protocol stack 1100 can be omitted, or othermodules added. The software modules can be run on a same processor ordifferent processors. Although only a few communication protocols arelisted, numerous wireless protocols can be used. For example, Bluetoothprotocols can include Basic Rate (BR), Enhanced Data Rate (EDR), and LowEnergy (LE) options. Bluetooth BR/EDR is also referred to as ClassicBluetooth.

The communication of data from a device (e.g., mobile device 115,companion device 120, first device 300, or second device 350) can occurthrough various protocols (e.g., 802.11 protocols, Bluetooth protocols,and near field communication (NFC) protocols). To determine whichprotocol to use, a device can include a link manager for determiningwhich protocol to use for a particular application, and thus whichdriver path data should be sent. A lower level link layer can alsoperform selections of a particular protocol to use. Further, a usertunnel (UTUN) controller can coordinate a plurality of virtualconnections with various client applications to communicate over acommon socket connection with another device (e.g., mobile device 115communicating with companion device 120).

In some embodiments, a client application 1105 on the device (e.g.,mobile device 115) can request data to be sent to another device (e.g.,companion device 120). The request can specify the other device via anysuitable identifier, e.g., an account name, an IP address, a MACaddress, etc. The request can be before or after the device determinesthat the other device is within communication, e.g., as determined byinitial signaling, such as a handshake. The data (e.g., in a message ora stream) can be sent any suitable application layer protocol, such asHTTP, RTP, SMTP, MGCP, etc. The other device can be any device,including another device of the user. The request can made be inresponse to an action by the user, an internal event (e.g., based ontime or other criteria) that may be in a same or other application(e.g., a calendar app), or an external event (e.g., in response to amessage from another device). An example of an event is a syncing event.

Before sending data, client application 1105 can submit an open socketrequest (e.g., in a streaming example). The socket request can useinformation from an identity services (IDS) framework 1115, which canprovide an address (or other type of ID) for the other device. Forexample, client application 1105 can know account information for thesecond device (e.g., account information of a different or same user),and IDS framework 1115 can store a list of device IDs for a particularaccount. IDS framework 1115 can be in communication with identitymanagement infrastructure 105 to obtain the list. Thus, IDS framework1115 can store or otherwise obtain device IDs (e.g., addresses) for alldevices that a user has registered with ID infrastructure 105. Forexample, IDS framework 1115 can request via an IDS daemon to IDinfrastructure 105 to obtain the device IDs. In one implementation, thesocket request can be made to kernel 1110.

In a messaging example, the request to send data can go to IDS framework1115 to obtain a device ID, which can be sent to message a messagecontroller 1118 and a user tunnel (UTUN controller 1120. UTUN controllercan establish a mapping between the device ID and an IP address (e.g., avirtual IP address) when the device ID is not an IP address. A socketcan be created between message controller 1118 (which assigns a deviceID to the socket) and kernel 1110 (which can assigns an address to thesocket, such as a virtual IP address). UTUN controller can be used tocreate the socket connection between message controller 1118 and kernel1110. In this manner, the send-date request from client application 1105does not need to include a device ID, but can specify an account, whichcan then be cross-referenced by IDS framework 1115 with known devices ofthe account and their capabilities (e.g., if the request requirescertain capabilities). Given that a device ID can be obtained, a pairingdoes not need to occur prior to creating the socket.

In various embodiments, IDS framework 1115 can receive a particularport/service at the other device from client application 1105, determinethe port/service based on information obtained from ID infrastructure105, or determine the port/service from a token sent in the request. IDSframework 1115 can then communicate a device ID and other headerinformation to message controller 1118 and/or UTUN controller 1120. IDSframework 1115 and UTUN controller 1120 can communicate via crossprocess communication (XPC). UTUN controller 1120 can be part of an IDSdaemon, and can receive a device ID from ID infrastructure 105.

As mentioned above, UTUN controller 1120 can create a virtual addressthat corresponds to the actual device address, where the virtual addresscan be used to create a virtual socket. A virtual socket can also becreated using any device ID (e.g., an actual address of a device orother ID). As an example, a socket can be created for communicationbetween client application 1105 and kernel 1110 (e.g., in a streamingcontext), where kernel 1110 can have various sockets open with variousclient applications. Kernel 1110 can have a single connection to UTUNcontroller 1120 for the other device and multiplex (mux) the data fromvarious client applications into the single connection. Instead or inaddition, UTUN controller 1120 can also perform the muxing, e.g., ifmultiple socket exist between kernel 1110 and UTUN controller 1120 forvarious client applications to the other device. Incoming data can bedemultiplexed (demuxed) for sending to the destination clientapplication.

As another example, a socket can be created between kernel 1110 andmessage controller 1118 (e.g., in a messaging context), where a socketcan be created for each destination device, with different sockets to asame device potentially having different priorities. Thus, a particularvirtual socket can be associated with a particular device and aparticular priority (e.g., high and low). Message controller 1118 canhave various connections to various client applications. Thus, messagecontroller 1118 can provide mux/demux capabilities.

UTUN controller can create a primary socket with the other device. WhenUTUN controller 1120 receives data using a virtual connection associatedwith the second device, it can then map the virtual connection to theprimary socket for communicating with the other device. All data for theother device can then be sent out through the primary socket. Thevirtual address for a virtual socket can be passed back to clientapplication 1115, e.g., in the stream context. In one embodiment, avirtual socket involving kernel 1110 is a TCP socket. The virtualaddress can have a same format as a regular address, e.g., an IPv6address. A mux module can include any combination of kernel 1110,message controller 1118, and UTUN controller 1120.

When client application sends data, client application 1105 can use thevirtual socket to send data to kernel 1110. For example, the data can besent using TCP via the virtual socket. Kernel 1110 can implement an UTUNinterface for communicating with UTUN controller 1120. Kernel 1110 wouldpass the data (e.g., with a TCP header) and the virtual socketidentifying the virtual address to UTUN controller 1120, which wouldthen use the virtual address to resolve the device address fordetermining the device socket.

When sending to the data over the device socket, a link manager 1125 candetermine which link to use. A link can be a particular combination of awireless interface protocol (e.g., Bluetooth or Wi-Fi), a transportprotocol (e.g., TCP, UDP, etc.), and a destination device. In thismanner, UTUN controller 1120 does not need to know how the data is beingsent, but instead can simply send the data to link manager 1125.

In various embodiments, the determination by link manger 1125 can bemade per data packet, per set of data packets, per device socket, andmay change from one data packet to another. Link manager 1125 may thenselect a link for sending the data. In the example shown, a Wi-Fi link1130 provides software drivers for communicating with one or more Wi-Fiprotocols, and BLTE link 1135 provides software drivers forcommunicating with Blutooth LE. Wi-Fi link 1130 is in communication withWi-Fi hardware 1160, and BLTE link 1135 is in communication with BTLEhardware 1155. Wi-Fi link 1130 can be used for various Wi-Fi protocols,such as infra-Wi-Fi (infrastructure Wi-Fi). In one embodiment, linkmanager 1125 can try all links to determine whether any of the links cancontact the other device, and then use a connected link with a highestpredetermined rank or dynamic rank.

Hardware 1150-1160 can be in communication with links assigned tovarious devices. For example, links 1130, 1135, and 1140 can be assignedfor communication with a second device. And, other links that areassigned for communication with a third device can also be incommunication with hardware 1150-1160. When a particular hardwarereceives data, software can identify a particular sending device andthen determine the corresponding link, e.g., using header information todetermine the link corresponding to the sending device and transportprotocol.

In some embodiments, a combined link 1140 can include an interface 1144for communicating with link manager 1124 and a selector 1148 thatselects a particular protocol to use. The protocols can be the same ordifferent than that available to link manager 1125. Selector 1148 canperform similar functions as link manager 1125 in that a particular linkis selected. However, link manager 1125 and selector 1148 can usedifferent criteria for determining which link to use. For example, linkmanager 1125 can determine to use combined link 1140, and selector 1148can then determine that BTLE hardware 1155 is to be used. The hardwarecan be contained on a same or separate chips.

One or more protocols can be only available via combined link 1140, suchas classic Bluetooth hardware 1150. Link manager 1125 and selector 1148can use various criteria for determining which link to use, such aspower usage of a link, speed of a link (e.g., real-time data rate), andsignal strength of a link. A goal of the optimization for selecting alink can be to provide a minimal data rate at a lowest possible energy.

While various embodiments of devices, systems, and methods are describedherein, it will be apparent that the structure and order of objects oraspects of a method may be performed or placed in different order whileremaining within the scope of the described devices, systems, andmethods for performing relay with a call.

What is claimed is:
 1. A method of using a wearable device associatedwith a user account to initiate a call via a mobile device, the methodcomprising: establishing, by the wearable device, a communicationconnection with the mobile device, the communication connectionconfigured to share data with the mobile device, the mobile deviceconfigured to act as host for the call and to relay an audio signalassociated with the call to the wearable device, via a call connection,after the call is answered by a recipient of the call, wherein themobile device is associated with the user account that is associatedwith the wearable device; receiving a recipient number for the recipientof the call; transmitting, by the wearable device, the recipient numberto the mobile device via the communication connection; and at least inresponse to receiving, via the communication connection, informationthat identifies that the recipient answered the call: establishing thecall connection for the call with the mobile device, the call connectionconfigured to share the audio signal between the wearable device and themobile device; and receiving, via relay through the call connection withthe mobile device, the audio signal associated with the call.
 2. Themethod of claim 1, wherein a request to initiate the call is transmittedthrough the communication connection.
 3. The method of claim 1, whereinat least one of the communication connection or the call connectioncomprise a Bluetooth connection.
 4. The method of claim 1, wherein themobile device comprises a cellular interface.
 5. The method of claim 1,wherein the wearable device is configured without a cellular interface.6. The method of claim 1, wherein the recipient number is received via atouch screen of the wearable device.
 7. The method of claim 1, furthercomprising receiving, by the wearable device, a selection of the mobiledevice.
 8. The method of claim 1, wherein the communication connectioncomprises a first connection.
 9. The method of claim 8, wherein the callconnection comprises a second connection that is different from thefirst connection.
 10. A wearable device associated with a user accountthat is configured to initiate a call via a mobile device, the wearabledevice comprising: a wireless interface; a display; and at least oneprocessor coupled to the wireless interface and the display, the atleast one processor configured to: establish, via the wirelessinterface, a communication connection with the mobile device, the mobiledevice configured to act as host for the call and to relay an audiosignal associated with the call to the wearable device, via a callconnection, after the call is answered by a recipient of the call,wherein the mobile device is associated with the user account that isassociated with the wearable device; receive a recipient number for therecipient of the call; transmit, via the communication connection, therecipient number to the mobile device; and at least in response toreceiving, via the communication connection, information that identifiesthat the recipient answered the call: establish the call connection forthe call with the mobile device, the call connection configured to sharethe audio signal between the wearable device and the mobile device; andreceive, via relay through the call connection with the mobile device,the audio signal associated with the call.
 11. The wearable device ofclaim 10, wherein the processor is further configured to: provide, onthe display, a call end interface; receive an input to end the call atthe call end interface; and close the call connection in response to theinput to end the call.
 12. The wearable device of claim 11, wherein thecall is configured to continue at the mobile device after the callconnection with the wearable device is closed.
 13. The wearable deviceof claim 10, wherein the processor is further configured to present calldetails on the display.
 14. The wearable device of claim 13, wherein thecall details comprise caller ID information.
 15. The wearable device ofclaim 10, wherein the communication connection is configured to sharedata with the mobile device.
 16. A non-transitory computer-readablestorage medium having stored thereon computer-executable instructionsthat, when executed by a processor of a wearable device associated witha user account and configured to initiate a call via a mobile device,cause the processor to perform operations comprising: establishing acommunication connection with the mobile device, the mobile deviceconfigured to act as host for the call and to relay an audio signalassociated with the call to the wearable device, via a call connection,after the call is answered by a recipient of the call, wherein themobile device is associated with the user account that is associatedwith the wearable device; receiving a recipient number for the recipientof the call; transmitting, via the communication connection, therecipient number to the mobile device; and at least in response toreceiving, via the communication connection, information that identifiesthat the recipient answered the call: establishing the call connectionfor the call with the mobile device, the call connection configured toshare the audio signal between the wearable device and the mobiledevice; and receiving, via relay through the call connection with themobile device, the audio signal associated with the call.
 17. Thenon-transitory computer-readable storage medium of claim 16, wherein theoperations further comprise: receiving a call end instruction, from themobile device, over the communication connection; and closing the callconnection in response to the call end instruction.
 18. Thenon-transitory computer-readable storage medium of claim 16, wherein thecall connection comprises a peer-to-peer connection.
 19. Thenon-transitory computer-readable storage medium of claim 16, wherein thecommunication connection comprises a short-range wireless connection.20. The non-transitory computer-readable storage medium of claim 16,wherein the communication connection is configured to share data withthe mobile device.